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What is claimed is: 

1. A machine learning method, for at least one of data regression and data 
classification, that is a hybrid of neural net ("NN") analysis and support vector 
machine ("SVM") analysis, the method comprising: 

(1) providing an NN component, having an input layer and a hidden layer 
and an input vector space, and providing an SVM component, having a feature 
vector space; 

(2) selecting a group of parameters and combinations of parameters and 
providing a feature space coordinate, in the feature vector space, for each 
selected parameter and selected parameter combination in the input space; 

(3) providing at least one vector of candidate parameter values for each of 
the group of parameters in the input space; 

(4) providing initial values for connection weights between the input layer 
and the hidden layer for the NN component; 

(5) computing hidden layer output signals, corresponding to the connection 
weight values, for each of the parameter value vectors; 

(6) determining an inner product value of a selected number of at least two 
feature space coordinates; 

(7) providing a Lagrange functional using the determined inner product 
values; 

(8) providing at least two constraints, expressed in terms of Lagrange 
multipliers and input space data; 

(9) minimizing the Lagrange functional, subject to at least one selected 
constraint, to obtain Lagrange multiplier values corresponding to the minimized 
Lagrange functional; 

(10) computing a training error, using the connection weights for the NN 
component and the Lagrange multiplier values for the SVM component; 
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(11) when the computed training error is greater than a selected threshold 
value, changing at least one of the connection weights and repeating steps (5)- 
(10); and 

(12) when the computed training error is not greater than the threshold 
value, interpreting the NN component with the associated connection weights and 
the SVM component with the associated Lagrange multipliers as a trained 
NN/SVM system. 

2. The method of claim 1, further comprising: 
providing an optimization method; and 

using the optimization method in at least one of steps (5) through (12) to 
minimize said training error and to obtain at least one of said connection weight 
values and said Lagrange multiplier values. 

3. The method of claim 1, further comprising determining an optimized 
design by applying a response surface analysis to said design, using said trained 
NN/SVM system. 

4. The method of claim 3, further comprising providing a selected 
optimization procedure in determining said optimized design. 

5. The method of claim 1, further comprising augmenting said inner 
product value with at least one user-specified inner product value to said SVM 
component. 

6. The method of claim 1, further comprising: 

providing a collection of N data points in an M-dimensional space for said 
input space, where M>2 and N>2, and where each data point is assigned an 
indicium associated with one of at least first and second mutually exclusive sets; 
and 
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applying the method of claim 1 for determination of a separation surface in 
the M-dimensional space that separates the data points into at least first and 
second mutually exclusive regions that contain substantially all data points in the 
first set and in the second set, respectively. 

7. The method of claim 6, further comprising providing a visually 
perceptible view of at least a portion of said separation surface in at least two 
dimensions. 



8. A method for design optimization, the method comprising: 

(1) providing a group of M parameters that define a design, and providing 
a vector p = pO of initial values for each parameter in the group, where M is a 
selected integer > 1; 

(2) providing data point values for an optimal design at one or more 
selected location values; 

(3) providing an M-simplex in parameter space, centered at a vector 
location p = pO and having a selected diameter dO; 

(4) providing a design function, and providing a data point value for the 
first design function at each location value, depending on the choice of the 
parameter vector p, for the parameter vector pO and for each parameter vector 
corresponding to a vertex of the M-simplex, and for an expanded M-simplex, 
centered at pO; 

(5) providing a selected first objective function OBJ(p;pO;l), dependent 
upon the parameter vector p and upon a difference between the optimal design 
data point value and the first design function data point value at one or more of 
the location values; 

(6) determining a parameter vector p = p(min) within the expanded M- 
simplex for which the first objective function attains a minimum value; 

(7) computing a selected second objective function OBJ(p;pO;2) for p = 
p(min) and for p = pO; 
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(8) when OBJ(p(min);pO;2) is not less than OBJ(pO;pO;2), providing a 
modified expanded M-simplex, with a modified diameter d' satisfying dO < d' < d 
and repeating steps (6) and (7) at least once; and 

(9) when OBJ(p(min);pO;2) is less than OBJ(pO;pO;2), determining if 
OBJ(p(min);pO;2) is no greater than a selected threshold value; 

(10) when OBJ(p(min);pO;2) is greater than the threshold value, providing 
a substitute M-simplex, centered at p = p'O = p(min) with the selected diameter 
dO, and an expanded substitute M-simplex, centered at p'O with the diameter d, 
and repeating steps (4), (5), (6), (7), (8) and (9) at least once; and 

(11) when OBJ(p(min);pO;2) is not greater than the threshold value, 
interpreting the parameter set p = p(min) as an optimal design set. 

9. The method of claim 8, further comprising choosing said first objective 
function to be 

K 

OBJ(p;p0) = X w k lf(r k ;p) - f^opi)!^ 
k=l 

where is one of said selected location values, f(r k ;p) is one of said design 
function data point values, ftr^opt) is one of said optimal design data point 
values, w k is a selected non-negative weight coefficient, q is a selected positive 
number, and K is a selected positive integer. 

10. The method of claim 8, further comprising choosing said first objective 
function to be the same as said second objective function. 

11. The method of claim 8, wherein said second objective function depends 
upon said second design function, upon said parameter vector p and upon a 
difference between said optimal design data point value and a data point value 
computed using computer simulation of a response of said design, at one or more 
of said location values. 
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12. The method of claim 8, further comprising choosing said design 
function to correspond to pressure on an airfoil at said selected locations on a 
perimeter of the airfoil. 

13. The method of claim 8, further comprising: 

providing data point values for a second optimal design at one or more 
selected location values; and 

applying steps (l)-(ll) of claim 8 to obtain an optimal set of design 
parameters for the second optimal design, where the second optimal design has 
selected third and fourth objective functions that are independent of said first and 
second objective functions. 



